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Using Data to Guide School Improvement 

A lthough certain provisions of the No Child Left Behind Act of 2001 remain con- 
troversial, there is broad agreement that measuring student progress and setting 
specific goals are fundamental to school improvement (Schmoker, 1999). How do 
educators measure progress and set goals? The process begins with data. 

The ability to track performance is as valuable for educators as it is for doctors, busi- 
nesspeople, and competitive athletes. Yet education leaders often lack formal training 
in data analysis. They may suffer information overload when they attempt to draw 
conclusions from dozens of variables. To remedy the situation, educators need a 
process for data-driven decision making that helps them focus on the essential pieces 
of information to identify priority areas and select realistic goals. 

This issue of Notes & Reflections provides a recipe for school improvement based on 
data exploration. The main ingredients are the data themselves: useful information 
about schools and students. The recipe has four steps: 

1. Prepare for data exploration. The process begins with promoting a data culture 
and forming a team that will examine the data. 



Welcome to Notes & Reflections, our 
newsletter for professional development 
providers. Notes & Reflections offers 
practical information — ideas, strategies, 
tools, and resources — about topics of 
special interest to professional devel- 
opers who are working to improve 
school performance. 

Today, professional development 
services to schools are provided by 
variety of people: teacher leaders; 
principals; and district, regional, and 
state education staff — to name just a 
few. If you are one of these providers. 
Notes <& Reflections will help you build 
your capacity to plan and implement 
powerful professional development. 



2. Collect and organize the data. The team identifies the relevant data, gathers it 
together, and organizes it for analysis. 

3. Analyze the data. The team reviews the data and looks for important patterns. It 
identifies specific programs or student groups most in need of improvement. 

4. Explain the results. The team uses data to generate hypotheses and explain the 
underlying causes of the areas needing improvement. 

Upon completion of these four steps, educators are prepared to use their data to 

inform school improvement efforts. 
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Using Data to Guide School Improvement 



STEP 1: PREPARE FOR DATA EXPLORATION 



istrict and school leaders are responsible for 
creating an organizational culture that values the 
use of data and for forming the team that will focus on 
data exploration. The professional developer can help 
direct their efforts. 

Promote a Data Culture 

In schools with a data-friendly culture, teachers and 
administrators seek reliable data to inform their decisions 
about curriculum and instruction. Fostering such an atmos- 
phere is a gradual process (Holcomb, 1999). A first activ- 
ity for professional developers is to explain the purpose of 
data: to identify goals for school improvement. A second 
activity is to encourage administrators and teacher leaders 
to reflect on how they make critical school decisions. 

An excellent resource for introducing educators to the 
practice of data use is the video and print package titled 
Data Exploration: A Journey to Better Teaching and 
Learning. It’s listed in Tools and Resources on page 13. 



Tip: Assess Your Data Savvy 



The following questions (adapted from Sargent, 

2003, p. 1 1) help to raise awareness of the need 

for data-driven decision making: 

1 . How do we currently establish our improvement 
goals each year (for the district, a school, or a 
program)? Describe the process. 

2. Generally describe the achievement levels of the 
students in your building or program. 

3. What academic areas are in greatest need of 
improvement? With what content areas within each 
subject are students having the most difficulty? 
On what basis do you answer these questions? 

4. In your opinion, what are the primary factors 
within the school's control that cause low achieve- 
ment in the areas of greatest need? What data 
have you examined to support your opinions? 

5. What do you hope that the school or district 
will accomplish by data-driven decision making? 



Also, see Additional Reading on page 13 for publica- 
tions that explain how and why to use data for school 
improvement. 

Form a Data Exploration Team 

Data exploration is a task that is best accomplished by a 
broad-based team of educators. A team effort is crucial to 
the success of data-driven decision making for the follow- 
ing reasons: 

• There is too much work for any one person to do. 

• Input from multiple points of view is useful in order to 
interpret the data and brainstorm solutions. 

• A broad-based approach builds consensus and buy-in, 
which helps sustain the decisions that are to be made. 

The experience of staff at Learning Point Associates is 
that the ideal data exploration team size is six or seven 
members. For each school, the team must include the 
principal and two to three teacher representatives from 
various grade levels. It is also desirable for the team to 
have a representative from special-education teachers and 
noncore teachers (e.g., the art or music teachers). A district 
representative on the team helps the school understand the 
resources its district can offer and helps the district keep 
in touch with the needs of the school. In addition, a tech- 
nology expert is quite helpful. 

Because membership on the data team can be a lot of 
work, consider rotating the membership on an annual 
or biannual basis. Members must be willing and able to 
meet regularly. 



Tip: Include a Technology 
Expert on the Team 



Many schools find it useful to involve the technology 
coordinator or whoever is experienced with spread- 
sheet software. The process of interpreting and 
graphing data goes much faster when one team 
member is familiar with features of spreadsheet 
applications such as data filters and pivot tables. 
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STEP 2: COLLECT AND ORGANIZE THE DATA 



T he second step is collecting and organizing the data. 

Four types of data can be used as indicators of 
a school’s success and progress: achievement data, 
demographic data, program data, and perception data. 
(See “Data Checklists” on page 4 for specific examples 
of critical data.) 

Types of Data 

Achievement data. Achievement data provide information 
on student learning and achievement. These data include 
standardized test scores, classroom-based assessments, 
rubric-scored writing portfolios, and class grades. 

Achievement data answer critical questions about student 
achievement: 

• What evidence shows the knowledge, skills, and under- 
standings our students have achieved? 

• What evidence shows the academic proficiency our 
students have achieved? 

Demographic data. Demographic data provide information 
about the characteristics of students: enrollment, student 
mobility, attendance, disabilities, ethnic background, limited 
English proficiency, economic status, gender, dropout 
rates, and behavioral problems. The purpose of this data is 
to ensure that the needs of all types of students are being 
met adequately and to indicate important demographic 
trends (e.g., an increase in English language learners). 

Demographic data answer critical questions about the 
student population: 

• Who are our students? What trends do we see in our 
student population? 

• What factors outside of school help us understand our 
students? 

Program data. Program data provide information on all 
school programs. These data represent the conditions 
directly under the control of the school, such as its cur- 
riculum scope and sequence, academic programs, teacher 
training and experience, professional development, after- 
school programming, and Title I programs. 

Program data answer the following critical questions: 



• What are our academic programs? Which students are 
involved in each? 

• Flow successful are our programs in attaining the aca- 
demic standards? 

Perception data. Perception data provide information 
on the attitudes and beliefs of various stakeholders in 
the school, such as teachers, students, and parents. Some 
categories for which perceptions are helpful include 
academic standards, school leadership, quality of instruc- 
tion, and school climate. 

Perception data are usually collected with surveys that 
address the following questions: 

• Flow do parents, teachers, students, and members of the 
school community feel about the job our school is doing? 

• Flow satisfied are our “customers” with our programs? 

In order to have longitudinal data, which identify trends 
over time, it is essential to find or create a good perception 
survey and to stick with it year after year. 

Organize Data 

Organization is the key to successful data use. The following 
activities will help your school assemble the data you need 
in a useful format. 

Set up a student-level file. Create a spreadsheet file of 
student-level data from this year’s state assessments. Each 
row will be a different student, and each column will be 
a different type of data. Start by labeling each column of 
data with a descriptive heading. Include in this file achieve- 
ment data as well as all of the demographic and program 
data you have for each student. Table 1 (on page 4) presents 
a sample student data file. (Note: When working with student 
data, it is important to maintain the confidentiality of the 
information. Most states, if not all, have laws that limit 
access to student data to school or district staff.) 

Organize by grade and topic. After you have created this 
basic file, the next activity is to organize it. Because of the 
sheer amount of data, a single file would be cumbersome. 
Therefore, it is helpful to break it into smaller files 
according to grade and topic. 
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Table 1. Sample Student Data File 



Student 


Title 1 


Free Lunch 


English Proficiency 


Academic Program 


Cohen, Jack 


Yes 


Yes 


Yes 


Vocational 


Ismaeli, Mustafa 


No 


Yes 


No 


Regular 


Smith, Maryanne 


No 


No 


Yes 


Gifted 



Data Checklists 

Checklist for Achievement Data 

REQUIRED 

□ State achievement test summary reports for the last 
three years for all grades tested. 

□ State achievement test student-level data for the last 
three years for all grades tested. 

□ School report card for the last three to four years. 

SUGGESTED 

□ Report card summaries: lists of students who are 
below a “C” in mathematics and language arts for 
Grades 3, 5, 8, and 11. 

• Include grade criteria descriptions. 

• Include lists for the last two to three years. 

□ Standardized test results (e.g., TerraNova, Iowa 
Tests of Basic Skills) with all reports, including 
school/district summary reports, objective/strand 
reports, and student-level data. 

Checklist for Demographic Data 

REQUIRED 

□ School report card that includes results for special- 
needs students and demographic groups. 

□ Demographic data such as gender, ethnicity, subsidized 
lunch eligibility, and English language learner status. 

SUGGESTED 

□ Lists of behavior/social referrals by grade. 

□ Student attendance data. 



Checklist for Program Data 

REQUIRED 

□ Lists of students participating in special programs 
such as gifted/talented, special education, and Title I. 

□ Summary of schoolwide professional development 
(i.e., staff development plan) for the past two to three 
years. Include any evaluations or survey results 
associated with the professional development offered. 

□ School improvement plan, both current and from 
the past two years. 

□ Dates of significant curriculum revisions (textbook 
implementation, realignment of course sequence) 
in core content areas. 

STRONGLY SUGGESTED 

□ Information about curriculum scope and sequence 
(e.g., curriculum maps). 

SUGGESTED 

□ Any program evaluation by external or internal 
evaluators. 

Checklist for Perception Data 

REQUIRED 

□ Any survey data, in summary form (preferably for 
multiple years). 

SUGGESTED 

□ Teacher absenteeism data. 
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STEP 3: ANALYZE THE DATA 



T he next step is to review and find patterns in the data 
that have been gathered. But before this step is begun, 
some preliminary planning will ensure that the team’s dis- 
cussions are productive. 

Before You Begin 

Reserve adequate time. Set aside a substantial block of 
time for the team to meet. Many schools have found success 
with a “data retreat” approach, in which they take a whole 
day to examine and discuss their data. 

Distribute the data. Upload the database onto the com- 
puters that your group will use during your discussion. 

You also may want to have the data in paper form. 

Gather necessary materials for displaying observations 
and conclusions. When you are ready to begin probing 
your data, have the following materials on hand to help 
you to display your data and conclusions: 

□ Highlighters (blue, green, yellow, and pink) 

□ Pencils 

□ Yardsticks and rulers 

□ Roll of masking tape (unless chart paper is self-stick) 

□ Markers for chart paper 
□ Flipchart easel 
□ Flipchart paper 

□ Laptop computer with Internet access 
□ Calculators 

□ Lunch, snacks, and beverages 

The Data Analysis Process 

With that preliminary planning completed, the team is 
ready to begin the process of data analysis. The goal is 
to identify strengths and weaknesses across the major 
academic areas, identify specific demographic groups that 
need academic improvement, and determine which programs 
are meeting or not meeting their goals. Throughout all of 
these analyses, there is a basic process for getting to the 
bottom line of the data. 



1 . Create a data summary table that lists the percentage 
of students who are proficient in certain topics across 
several years. 

2. Color-code the data to highlight areas of strength and 
weakness. 

3. Graph the data. 

4. Record and discuss the major findings. 

Use this process for each of the four types of data: 

achievement, demographic, program, and perception. 

Examine Student Achievement Data 

The first and most essential part of the process is identify- 
ing areas of strength and weakness in student achievement. 

The following steps can help the team examine student 

achievement in broad subject areas. 

1 . Find your broad district and school data summaries 
from the state assessments. 

2. Locate the report that shows the percentage of students 
at each level (e.g., below basic, basic, proficient, and 
advanced). Focus on the percentages of all students 
enrolled who are proficient in a given topic. Do not 
exclude special education or other exempted students, 
since the objective is to analyze data for all students 
enrolled. (Note: For some states, proficiency levels 

are listed as ranges of “scale scores,” so you might 
need to look for the percentage of students in each scale 
score range.) 

3. Calculate the percentages of all students at or above 
the proficient level for each grade and subject. Create a 
flipchart that lists the data for the past two to four years. 
Having data from the past four years will allow your data 
exploration team to compare how the same cohort of 
students (e.g., the class of 2006) performs across different 
grade levels, as well as how different cohorts of students 
perform at the same grade level (e.g., how the fourth 
grade performs across several years of students). Table 2 
focuses on one grade level for the past four years: 
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Table 2. Sample Data Chart: Grade 3 Proficiency Levels, 2000-04 





Scale Scores 


Grade 


School Year 


Reading 


Math 


Science 


Social 

Studies 


Writing 


3 


2001-01 


55% 


71% 


45% 


56% 


77% 


2001-02 


67% 


63% 


48% 


45% 


83% 


2001-03 


80% 


59% 


59% 


58% 


89% 


2001-04 


82% 


74% 


56% 


65% 


91% 



4. Use the stoplight method — with blue, green, yellow, and 
pink highlighters — to create a color-coding scheme indi- 
cating visually how well the student results are meeting 



your expectations. Table 3 defines the meaning of the 
colors. Your school may choose different cutoff points 
depending on your expectations. 



Table 3. Sample Color-Coding Scheme: Grade 3 Proficiency Levels, 2000-04 


Highlight 

Color 


Meaning 


Percentage of 
Student Proficiency 
(cutoff point) 


Blue 


WOW! Beyond expectations. 


80% and above 


Green 


GOOD! Meets expectations. 


70% to 79% 


Yellow 


CAUTION! Below expectations. 


60% to 69% 


Pink 


URGENT! In need of immediate improvement. 


0% to 59% 



5. Highlight the scores. Use the highlighters to color over 
each cell of the chart according to this coloring scheme. 



Table 4 shows the end result: a striking visual display 
that indicates areas of academic strength and weakness. 



Table 4. Sample Highlighted Data: Grade 3 Proficiency Levels, 2000-04 



Grade 


School Year 


Reading 


Math 


Science 


Social 

Studies 


Writing 


3 


2000-01 


55% 


71% 


45% 


56% 


77% 


2001-02 


67% 


63% 


48% 


45% 


83% 


2002-03 


80% 


59% 


59% 


58% 


89% 


2003-04 


82% 


74% 


56% 


65% 


91% 
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6. Display the data. For each grade, create a graph to 
display the proficiency level over several years in each 
academic area. Graphing makes data more tangible and 
communicable. 

There are several techniques for using graphs to look 
at longitudinal data. Here are two popular views: 



• Follow cohorts of students across several grades. 
This view indicates whether teaching effectiveness 
within a grade level is improving over time. 

In Figure 1, the fourth-grade level appears to be growing 
in its effectiveness with reading comprehension much 
more than the other grade levels. 
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Figure 1. Sample Graph: 

Reading Comprehension Across Year and Grade 
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Compare the same grade level over time, across 
different subjects. This view focuses on how 
instruction in various subjects is improving over 
time at a particular grade level. 



Figure 2 displays the percentage of students who met 
proficiency standards for eighth-grade math and reading 
across several school years. The graph shows that reading 
proficiency is improving over time, yet math proficiency 
rates are somewhat stagnant. 



Figure 2. Sample Graph: 

Grade 8 Proficiency in Math and Reading 




Reading 

Mathematics 
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Tip: Display Graphs on Chart Paper 



Putting your graphs on chart paper will enable the 
entire team to view and discuss the results together. 

7. Discuss and record major patterns. When you have 
finished creating the charts and graphs, consider the 
following questions: 

• What patterns do we see in this year’s results? 

Do we see similar patterns in past years’ results? 

• What trends emerge over the past several years? 

Do these trends indicate that we are moving toward 
our goals? 

8. Conclude by making a list of summary statements that 
describe the major findings. Write the data summary 
statements on chart paper. 

Examine Demographic and Program Data 

Although the broad analysis of achievement data is useful 
for describing general student strengths and weaknesses, 
it is necessary to delve deeper to understand what these 
results mean. To do so, look at breakdowns of your data 
according to demographics and program participation. 

All state assessments now disaggregate a school’s results 
according to major demographic categories. 

Create data charts to summarize these data. List the major 
demographic variables in the rows and the major subject 



areas in the columns. (Each chart can represent a specific 
grade and year.) Color-code each data point, and graph 
your important findings. 

Most achievement data, however, are not disaggregated by 
demographics or program participation. The state reports 
can reveal that low-income students show low proficiency 
in mathematics, for example, but it is necessary for each 
school to determine the areas of math in which these 
students need particular help. Use the following steps as 
a do-it-yourself guide to disaggregating data: 

1. Locate your student-level datafile (which was described 
in Step 2 in the section “Organize Data”). Color-code 
the achievement scores for each student, so that scores 
considered “excellent” receive one color, “good” scores 
receive another color, and so on. 

2. Sort the data or use filters to look at the students in 

a specific category. For example, you first can display 
only those students who received Title I assistance, 
and then view only those students who have not. The 
color coding will enable you to compare these two 
groups quickly. 

3. Calculate the percentage of proficient students for im- 
portant demographic and program categories. To get a 
longitudinal view, repeat this process for different years 
and different grades. 

4. Create charts and graphs of the data. Table 5 and Figure 
3 display the percentage of students in different academic 
programs who were proficient in ninth-grade math. 



Table 5. Sample Data Chart: Grade 9 Math Proficiency 


Program 


2001 


2002 


2003 


2004 


Accelerated 


94% 


99% 


99% 


99% 


Regular 


66% 


65% 


81% 


84% 


Vocational 


46% 


48% 


59% 


61% 
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Figure 3. Sample Graph: 
Grade 9 Math Proficiency 




2001 2002 2003 2004 



5. Look for patterns in the data. Discuss these patterns and 
list the observed strengths, weaknesses, and other unique 
patterns. For example, Figure 3 graphically shows a 
distinct jump in achievement starting in 2003 for both 
regular and vocational students. Use chart paper to write 
down the important patterns that you observe. 

Examine Perception Data 

Perception data verify that stakeholders such as teachers, 
parents, and students are satisfied with the way the 
school is being run. The guiding question is whether or 
not the stakeholders of the school, along with the staff, 
are satisfied with the way that the school operates. 
Examine perception data through a similar process as 
with the content areas: 

1. Create frequency tables. School surveys often ask people 
to express their degree of agreement or disagreement 
with statements about a school, such as “I feel safe in 
our school at all times.” Calculate the percentage of 
respondents who rated the school as satisfactory or better. 
In this example, the group would consist of the respon- 



dents who agreed or strongly agreed with the statement 
about school safety. 

2. Chart the responses. Create a chart for all of the items 
on the survey to display the percentage of respondents 
who rated the item as satisfactory or better. Table 6 
compares the responses of different groups. 

3. Color-code the results using the stoplight method. For 
each item, the team should decide what level of agree- 
ment is expected, better than expected, and worse than 
expected. Then it is possible to color-code each box 
using the stoplight method described earlier. 

Variation: If you have too much data or not enough 
time to analyze each item independently, just calculate 
an overall mean for each major section of your survey 
(e.g., Vision, Climate, Diversity). Then determine if the 
mean meets or exceeds your team’s expectations. 

4. Discuss and record patterns in the data. As survey data 
are shared and studied, team members should discuss 
and summarize the patterns they see. Write data sum- 
mary statements on chart paper. Major problem areas 
as well as demonstrated strengths should be listed. 
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Table 6. Sample Data Chart: Perceptions of School Climate 




Percentage of Respondents 
Who Indicated "Satisfactory" or Better 


Item 


Parents 


Students 


Teachers 


Teachers let parents know how well their children 
demonstrate the core values. 


68% 


61% 


88% 


All students and staff participate in creating a 
positive school climate. 


55% 


70% 


68% 


Diversity is celebrated through a variety of 
programs and activities. 


43% 


59% 


74% 



Summarize the Analyses 

Now it’s time to pull together all of your team’s observa- 
tions. The goal is to summarize and prioritize the school’s 
strengths and problems that have emerged from the data. 
Before you begin, appoint a discussion leader. Then tape 
all of the graphs and data summary statements to the walls. 

1 . Summarize the observed strengths, and compile them 
on large paper for all to see. Acknowledge and celebrate 
all of the school’s accomplishments. 

2. Brainstorm how to publicize these strengths to the 
community. Remind the team that solid data is highly 
effective for substantiating claims of school success. 



3. List all the problem areas that appeared in the data 
analysis. Use group processes to come to agreement on 
the list, and write it on large paper for all to see. 

4. Delete from the list all problems that cannot be 
influenced (e.g., student mobility), and those that 
were not clearly based on data. Review the remain- 
ing problems to be sure they are stated completely 
and clearly. 

5. Have each team member rank-order the problems in 
terms of urgency, with the most urgent getting a rank 
of 1 and so on. Tally the rank orders to create a final list 
that takes into account everyone’s rankings. 



Tip: Publicize Your Successful Results to the Community 



Data are the best and most credible way to persuade your community of the success of a particular program 

or the school as a whole. The following suggestions are helpful for publicizing this success to your community: 

1 . Use graphs to display successful programs. Post charts displaying year-by-year trends in your school's 
hallways. Include graphs in your school's newsletter and on its Web site. 

2. Target a message to Parent Night by displaying a chart of findings or having a speaker announce successes 
to the whole group. For example, administrators at Scott- Morgan Community Unit School District 2 in Bluffs, 
Illinois, observed that students with good attendance had higher achievement. In order to drive the message 
home, administrators displayed a large chart of the finding during a recent Parent Night. 

3. Celebrate. Your teachers and students have worked hard to reach your achievement goals. Celebrate the 
success with a schoolwide assembly. 

4. Publicize success in local newspapers. The district office can send a press release to the local newspaper 
announcing the success of a program. 
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STEP 4: EXPLAIN THE RESULTS 



tep 3 answered the question of “Where are we now?” 
The challenge of Step 4 is to determine “How did 
we get here?” or, more precisely, “Why are our students 
performing the way they are?” In other words, it’s time 
to find explanations for the results. For example, there 
may be a number of possible reasons for low math 
achievement, such as the inadequate alignment of text- 
book to standards, the lack of instructional time, or gaps 
in the curriculum. 

The key idea is for team member to meet, ask questions 
about the data, and discuss possible answers. Whenever 
possible, team members should consult additional data to 
corroborate their answers. 

1. Establish ground rules for discussion. It is important 
to maintain a safe and respectful environment that is 
conducive to discussing professional effectiveness. 



Ground Rules for Discussion 

□ The opinion of every team member matters. 

□ If you think it, say it — respectfully. 

□ All comments should be brief 
(less than one minute). 

□ Silence = consensus. 

□ One person talks at a time. 

O Respectfully listen to each member's hypothesis. 



2. Assign a recorder for the group discussion. This person 
will write down the hypotheses as they are suggested. 

3. Generate hypotheses. Considering one problem state- 
ment at a time (e.g., declining math achievement), team 
members take turns stating a hypothesis for it. The 
recorder writes down each hypothesis in the left column 
of the Hypothesis Worksheet (see Table 7 on page 12). 
Hypotheses frequently fall into the following categories: 

• The curriculum or curriculum materials do not align 
with the standards or the assessments. 

• Teachers have not received adequate professional 
development to teach to the standards effectively. 



Tip: Analyze Root Causes 



Identifying the root cause of problems leads to more 
strategic, effective interventions. A number of group 
processes are available for root cause analysis. One 
of these, the Fishbone Group Process, is described 
in Notes & Reflections Issue 4 (www.ncrel.org/info/ 
notes/spring03/fish.htm). For an additional resource 
on analyzing root causes, see School Leader's Guide 
to Root Cause Analysis, listed on page 1 4 of the 
Tools and Resources. 

• A significant number of students are at risk or have 
special needs that are not being met adequately. 

4. Evaluate the hypotheses. Use data such as curriculum 
maps, textbook adoption records, demographic trends, 
records of behavioral problems, and perceptions of 
various stakeholders to evaluate the hypotheses for 
each problem statement. In the right column of the 
Hypothesis Worksheet (see Table 7 on page 12), write 
evidence that supports or contradicts each hypothesis. 
Then record the final action: Accept as possible any 
explanation that is supported by evidence or reasoning; 
reject any explanation that is not. 
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Table 7. Sample Hypothesis Worksheet 


Problem Statement: Achievement levels in math drop grade-by-grade, until they are at very low levels in 
Grades 6, 7, and 8. They pick up slightly from Grades 9 to 12. 


Hypothesis 


Action and Evidence 


Our standards are too high. The achievement tests 
are too difficult, and get harder each year. 


REJECT. Students in neighboring districts, who take 
the same tests, do not show a decline in scores as the 
grades progress. We also have studied the items and 
concur that they are grade-appropriate. 


Our math teachers in the intermediate levels have 
not had the appropriate training to teach to the 
standards. 


ACCEPT AS A POSSIBILITY. We looked at the 
licensure, and our teachers do have the appropri- 
ate credentials. However, we also examined the 
sequence of professional development and there 
has been none in the area of mathematics for the 
past 10 years. 


We have too many students from low-income families 
who are not prepared to achieve at higher levels. 


REJECT. Students who are classified as economically 
disadvantaged do not perform any worse than 
other students. 


We have gaps in our curriculum. 


ACCEPT AS A POSSIBILITY. We examined our 
curriculum scope and sequence, and found that we 
spend very little time covering the objective areas 
on which our students fared poorly. 



Adapted from Using Data to Bring About Positive Results in School Improvement Efforts (North Central Regional Educational Laboratory, 2001, p. 41). 



NEXT STEPS: PROCEEDING FROM DATA TO ACTION 



U p to now, this issue has described four critical tasks: 
preparing for data exploration, collecting and 
organizing the data, analyzing the data, and explaining 
the results. This knowledge will serve as a powerful 
motivator for school change. As Ralph Grimm, super- 
intendent of the Canton Union School District 66 in 
Canton, Illinois, notes, “That chart [of data] is hard to 
argue against.” 

The challenge for your data exploration team is to turn this 
momentum into action. The next steps in the process are to 
set concrete goals, develop strategies for achieving those 



goals, determine how to evaluate progress, and roll out the 
plan. This series of steps is the second half of the process 
of data-driven decision making. It will be the focus of a 
forthcoming issue of Notes & Reflections. 

For additional hands-on experience using data, school teams 
might be interested in participating in a Data Retreat. 
Refer to “Spotlight on Learning Point Associates Data 
Retreats” on page 14. Also, the interested reader is directed 
to Using Data to Bring About Positive Results in School 
Improvement Efforts, a data tutorial that is available online 
(www.ncrel.org/toolbelt/tutorial.pdf). 
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an unsupportive school culture, lack of understanding about the importance of data, 
and lack of training in the use of data. It clarifies the importance of data, discusses 
what data to gather, suggests how to use and analyze data for school improvement, 
and indicates how to communicate about data and the results of analyses. The book 
can be ordered from the publisher (www.eyeoneducation.com). 
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interest in exploring data. 



Tools and Resources 



The following tools and resources facilitate the different steps 
of data exploration. 

Resource for Introducing a School to Data Exploration 

Data Exploration: A Journey to Better Teaching and Learning 

This multimedia package contains a 20-minute video featuring two schools that 
have used data and data-based processes to build and maintain school cultures 
of continuous improvement. In the video, educators and administrators in each 
school clearly articulate the ways they use data to improve student achievement. 
The package includes a booklet of three activities to start conversations around 
data use in school and the classroom. Ordering information and additional 
details are available through the Learning Point Associates product catalog 
(www2.1eamingpt.org/catalog/cart/item.asp?productID=93). 

Resources for Data Collection and Organization 

Quality School Portfolio 

This free Web-based tool helps schools and districts make data-based decisions. 
It consists of two free software applications — a Data Manager and a Resource 
Kit — that offer schools a solution for collecting and storing student data that 
can best inform their practices. The Data Manager is a database that enables 
users to disaggregate, import, and analyze data. The Resource Kit contains 
surveys, questionnaires, and observation protocols to help schools gather 
information about their school’s climate and instructional practices, safety 
and security, parent involvement, professional development, curriculum and 
instruction, technology and innovation, and special programs. It is available 
online (qsp.cse.ucla.edu). 

Characteristics of Successful Schools Web Site 

This Web site, maintained by the Wisconsin Department of Public Instruction, 
contains several surveys that can be used to collect perception data. Although 
only Wisconsin public schools can administer surveys from the site, the surveys 
themselves are available for viewing online (goal.ncrel.org/winss/sample.htm). 



School and District Report Cards 

Many states and districts have created Web-based reporting systems to provide 
school- and district-level data to constituents. The Learning Point Associates 
ToolBelt Web site contains a Web page of “Regional, State, and District Report 
Cards” with links for Illinois, Indiana, Iowa, Michigan, Ohio, and Wisconsin. 
This page is available online (www.ncrel.org/toolbelt/states.htm). 

Resources for Explaining the Data 

The following resources help to explain the underlying causes of student 
achievement patterns, with a focus on curriculum scope and sequence: 

Surveys of Enacted Curriculum 

The Surveys of Enacted Curriculum (SEC) are a research-based tool that 
collects and reports data on what content is taught and how it is taught in 
schools and districts. The SEC compare this enacted curriculum to what is 
expected by standards and assessments. Additional information is available 
at the SEC Web site (www.secsupport.org). 

Notes & Reflections, Issue 4 

This issue of Notes & Reflections outlines a process for curriculum mapping. 

It includes the Fishbone Group Process, a tool for analyzing root causes. The 
entire issue is available online (www.ncrel.org/info/notes/spring03/npsp03.pdf). 

Guide for Instructional Leaders, Guide 1: An ASCD Action Tool 

This 215-page resource by Roland Barth, Bobb Darnell, Laura Lipton, and 
Bruce Wellman describes a process of continuous educational improvement 
and includes tools for analyzing root causes. It is available from the Association 
for Supervision and Curriculum Development (shop.ascd.org). 

School Leader’s Guide to Root Cause Analysis 

This book by Paul Preuss explains the concept and the process of root cause 
analysis and includes numerous real-life examples in the area of school 
improvement. It can be ordered from the publisher, Eye on Education 
(www.eyeoneducation.com). 
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Spotlight on 

Learning Point Associates Data Retreats 

T he most common barrier to using data in schools is 
the lack of time to collect, organize, analyze, and 
interpret the information. Educators are often faced with 
the challenge of conducting data review in short meetings 
before or after school or as a smaller part of a professional 
development day. For this reason, many schools are opting 
for a Data Retreat approach. Instead of tackling the task 
piecemeal, a Data Retreat is a dedicated block of time 
during which a school team accomplishes the major tasks 
of data-driven decision making. 

Learning Point Associates conducts Data Retreats on-site 
at schools and districts. According to Bryan Chumbley, 
senior professional development associate at Learning 
Point Associates, conducting a Data Retreat early in 
the improvement process helps schools examine their 
data history and clarify their future improvement path. 
“When schools examine data that tell the story of where 
they have been and where they are, they are better 
prepared to make decisions about where they want to 
go,” Chumbley says. 

One of the first challenges uncovered in a Data Retreat 
is identifying what data to collect and deciding how to 
organize it. Sometimes schools struggle because they have 
no data or what they have is inconsistent. To address this 
concern, Chumbley conducts a preliminary meeting with 



school leaders in order to identify and organize the data 
that they will bring to the retreat. 

Another critical issue is appropriate training. “Educators 
in general receive inadequate training in the areas of data 
use and analysis,” says Chumbley. “Even when they are 
provided a block of time to review data, feelings of being 
ill-prepared to engage in such work are pervasive.” 

Data Retreats provide a unique opportunity to address this 
common barrier. Chumbley describes the two-day session 
as a time to receive training in processes to analyze and 
interpret data, as well as an opportunity to engage in those 
processes to review school data. “Participants are able to 
see the data transformed into meaningful charts and 
graphs, which make trends and patterns apparent,” he 
explains. “This kind of information benefits decisions 
about curriculum planning and instruction.” 

Note: Portions of this section originally appeared in What’s Working: 
Data-Driven Decision-Making in the Schools from LRP Publications. 
Reprinted with permission. (For more information on What’s Working , 
call 800-341-7874 or visit online at www.whatsworkingdata.com.) 



To obtain more information about Learning Point 
Associates Data Retreats, to schedule a Data 
Retreat for your school or district, or to learn how 
Learning Point Associates can serve your school 
or district's professional development needs, 
call the Learning Point Associates Service and 
Product Order Line at (800) 252-0283. 



For more information, contact: 

LEARNING POINT 

Associates™ 



P: 630.649.6500 > F: 630.649.6700 

800-356-2735 > www.learningpt.org 
1 120 East Diehl Road, Suite 200 
Naperville, IL 60563-1486 
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